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SUMMARY 

In  what  follows  we  wish  to  present  a  potpourri  of  problems 
of  some  Interest  and  difficulty  arising  in  the  field  of  nonlinear 
circuit  theory.  Perhaps  the  only  sensible  way  to  catalogue 
scientific  problems  is  in  terms  of  "solved  or  unsolved."  Yet 
this  classification  is  itself  a  very  subjective  one,  dependent 

/ 

upon  the  times  and  the  fashions.  Recall  the  dictum  of  Poincare 
that  the  solutions  of  one  generation  are  the  problems  of  the 
next . 

In  what  follows,  we  have  for  the  sake  of  convenience 
attempted  to  group  categories  of  problems  under  the  headings 
of  "descriptive,"  "control,"  "stochastic,"  and  so  forth.  Con¬ 
venient  as  some  of  this  nomenclature  is,  it  should  be  regarded 
with  a  certain  amount  of  suspicion.  Most  significant  problems 
blithely  cut  across  these  artificial  boundaries  within  fields 
of  specialization,  and  within  science  itself. 

In  these  days  of  rapidly  and  dramatically  changing  tech¬ 
nology,  it  would  be  rather  brash  to  attempt  to  predict  the  type 
of  mathematics  that  will  be  most  urgently  required  even  ten 
years  from  new.  It  is,  however,  fairly  safe  to  look  about  and 
note  the  requirements  of  the  present  and  of  five  years  back. 

The  difficulties  that  abound  render  a  certain  time  lag  inevit¬ 
able,  and  it  may  well  be  that  new  scientific  developments  may 
render  fields  obsolete  and  matnematical  solutions  for  problems 
within  those  fields  unnecessary  before  they  are  even  obtained. 
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DIRECTI0N3  OP  MATHEMATICAL  RESEARCH 
IN  NONLINEAR  CIRCUIT  THEORY 

Richard  Bellman 

INTRODUCTION 

In  what  follows  we  wish  to  present  a  potpourri  of  problems 
of  some  interest  and  difficulty  arising  in  the  field  of  non¬ 
linear  circuit  theory.  Perhaps  the  only  sensible  way  to 
catalogue  scientific  problems  is  in  terms  of  ’’solved  or 
unsolved."  Yet  this  classification  is  itself  a  very  subjective 
one,  dependent  upon  the  times  and  the  fashions.  Recall  the 
dictum  of  Poincare  that  the  solutions  of  one  generation  are  the 
problems  of  the  next. 

In  what  follows,  we  have  for  the  sake  of  convenience 
attempted  to  group  categories  of  problems  under  the  headings 
of  "descriptive,"  "control,"  "stochastic,"  and  so  forth. 
Convenient  as  some  of  this  nomenclature  is,  it  should  be  re¬ 
garded  with  a  certain  amount  of  suspicion.  Most  significant 
problems  blithely  cut  across  these  artificial  boundaries 
within  fields  of  specialization,  and  within  science  Itself. 

In  these  days  of  rapidly  and  dramatically  changing  tech¬ 
nology,  it  would  be  rather  brash  to  attempt  to  predict  the 
type  of  mathematics  that  will  be  most  urgently  required  even 
ten  years  from  now.  It  Is,  however,  fairly  safe  to  look  about 
and  nc te  the  requirements  of  the  present  and  of  five  years 
back.  The  difficulties  that  abound  render  a  certain  time  lag 
inevitable,  and  it  may  well  be  that  new  scientific  developments 
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may  render  fields  obsolete  and  mathematical  solutions  for 
problems  within  those  fields  unnecessary  before  they  are  even 
obtained. 

We  have  made  no  attempt  to  present  all  of  the  fields  of 
mathematical  research  that  stretch  ahead  so  invitingly.  Rathei , 
we  have  fastened  our  attention  upon  those  that  seem  most 
Intriguing  or  most  significant  In  our  own  eyes,  and  those  in 
which  we  have  meandered  to  some  extent  or  other. 

Finally,  we  make  no  pretense  of  knowing  how  to  tackle 
most  of  the  major  problems  we  pose.  If  anything,  we  may 
conclude  that  new  aoproaches  are  vitally  needed  at  the  present 
time  to  tackle  the  truly  formidable  problems  we  face.  There 
Is,  in  consequence,  an  enormous  opportunity  for  the  original 
mind  to  made  fundamental  contributions. 
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DESCRI PTIYE  PRO CESJEE 


1,  Descriptive  Processes 

de  shall  consider  two  essentially  different  types  of 
processes.  One  we  shall  call  descript  1 ve  and  the  other  control . 
In  the  first  case,  we  3hall  consider  a  physical  system  whose 
state  1 8  described  by  a  functional  equation  of  the  form 

(1)  =  F(x),  x(0 )  =  c. 

Here  x  Is  an  N-dlmenslonal  vector  and  F(x)  Is  a  functional 
of  x(t).  The  problem  Is  then  that  of  determining  the  behavior 
of  x(t)  over  all  time.  This  field  contains  the  classical 
theory  of  differential  equations,  and,  as  we  shall  see,  much 
mo  re . 

Subsequently,  we  shall  define  what  we  mean  by  a  control 
process  In  more  precise  fashion.  Meanwhile,  let  us  think  of  a 
process  of  this  nature  as  one  In  which  we  alter  the  structure 
of  the  system  In  various  ways  In  order  to  obtain  a  more 
desirable  behavior  of  the  system  over  time. 

Within  the  category  of  descriptive  processes,  we  shall 
make  further  splittings  Into  determlnl stl c  and  stochastic , 
linear  and  nonlinear,  and  finally,  low-  and  hlgh-dlmenslonal . 

It  Is  not  hard  to  believe  that  we  could  devote  a  paper  of 
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this  size  and  more  to  any  one  of  these  subdivisions.  Conse¬ 
quently,  if  we  appear  to  skim  and  dart  or  to  stroll  In  seven 
league  boots,  we  hope  that  the  reader  will  forgive  us. 

2.  Differential  Equations 

Let  us  begin  with  the  traditional  subject  of  differential 
equations.  Let  x  be  an  N— dimensional  vector  determined  by 
the  vector  differential  eqjation 

(1)  jjr  -  g(x),  X'O)  -  C. 

The  basic  challenge  Is  that  of  predicting  the  behavior  of  x(t) 
as  t  Increases.  This  is  the  classical  descriptive  process. 

For  some  reasons  which  we  shal3  enter  Into  below  this  is  a 
never-ending  challenge  which  in  all  probability  will  never  be 
completely  answered.  However,  for  reasons  which  we  shall  also 
describe  below,  it  may  be  a  challenge  to  which  we  may  not  wish 
to  respond . 

In  the  following  sections,  we  shall  discuss  linear  and 
nonlinear  differential  equations,  analytic  and  computational 
aspects,  deterministic  and  stochastic  versions,  and,  finally, 
some  aspects  of  the  curse  of  dimensionality. 

j.  Linear  Differential  Equations — Analytic  Aspects 

The  benavior  of  the  solution  of  a  differential  equation  is 
quite  readily  determined  when  we  possess  some  explicit  repre¬ 
sentation  such  as  sinfcjt  +  ,  or  ^(t),  or  even  a  contour 


Integral  such  as 
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.  4 

(1)  u(t;  -  /'  e2^2  dz. 

In  general,  differential  equations  introduce  new  functions 
which  cannot  be  represented  in  terms  of  the  familiar  trans¬ 
cendents  of  Whittacker  and  Watson. 

As  simple  an  equation  as  the  second  order  linear 
differential  equation 

(2)  a"  +  a(t)u  -  0, 

cannot  be  integrated  in  terms  of  quadratures  and  the  usual 
functions  of  mathematical  physics  when  a(t)  is  a  general 
function;  cf .  [1]  .  Furthermore,  even  if  a(t)  has  a  quite 
simple  form,  there  is  no  guarantee  that  the  solutions  will 
possess  any  simple  structure.  Thus  the  equation 

(3)  u"  +  ( a  +  b  cos  t)u  ■  0, 

the  Mathleu  equation,  presents  many  intricacies  of  analysis, 
and  essentially  has  a  theory  of  its  own. 

It  follows  that  new  techniques  must  be  devised  to  enable 
us  to  derive  information  concerning  the  nature  of  the  solution 
directly  from  the  properties  of  the  coefficients,  without  the 
intervention  of  explicit  analytic  solutions.  An  enormous 
amount  of  effort  ha3  been  expended  in  this  activity,  stimulated 
by  the  classical  Sturm-Llouvllle  problems  of  mathematical 
physics,  and  by  the  newer  Schrodlnger  equation;  see  [2] ,  |jj  , 
for  many  references  . 
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The  study  of  linear  systems  is  of  particular  importance  in 
conjunction  with  stability  analysis,  [ 2 ] .  Leaving  aside  the 
case  of  constant  coefficients,  by  no  means  a  trivial  case  as  we 
shall  see  below,  the  next  two  most  important  types  of  linear 
equations  to  consider  are  those  with  periodic  coefficients  and 
those  with  almost-periodic  coefficients. 

Let  us  discuss  these  equrtions  quite  briefly.  Por  the 
case  of  periodic  coefficients,  the  result  of  Ploquet  yields  a 
foothold  which  can  occasionally  be  further  exploited.  Given 
the  equation 

( 4  )  «  A  ( t )  x , 

where  A(t)  is  periodic,  say  of  period  1,  the  aforementioned 
result  asserts  that  every  solution  of  (4)  has  the  form 

(5  )  x  -  eBtp ( t ) , 

where  p(t,  is  a  periodic  vector  with  period  1  and  B  is  a 
constant  matrix.  This  is  an  existence  theorem  which  of  Itself 
furnishes  no  Information  as  to  the  nature  of  B. 

Any  results  beyond  this  are  come  by  hard.  Por  an 
extensive  survey  of  results  and  techniques,  see  P*] .  Let  us 
make  one  remark  in  thiB  connection.  Suppose  that  A(t)  ia  not 
only  singly-periodlc  in  the  real  variable  t,  but  doubly— 
periodic  as  a  function  of  the  complex  variable  t.  In  this 
case,  one  can  theoretically  obtain  an  explicit  representation 
for  x  in  terms  of  doubly— periodic  functions.  This  result  of 
Hermite,  ,  has  not  been  exploited.  It  seems  reasonable  to 
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suppose  that  a  good  deal  of  information  concerning  tne  Mathleu 
functions  and  related  functions  can  be  obtained  using  the  fact 
tnat  the  Jacobian  elliptic  functions  reduce  to  the  usual  sine 
and  cosine  as  the  modulus  approaches  zero. 

For  the  case  of  almost-perlodic  coefficients,  the  situar- 
tlon  is  very  discouraging,  despite  tne  determined  endeavors  of 
a  number  of  mathematicians .  A  little  known  paper  by  Shtokalo, 
[6]  ,  contains  some  interesting  results  which  seem  capable  of 
extension,  but  apart  from  this  there  are  few  leads.  It  would 
seem  that  the  class  of  almost-perlodic  functions  is  too  wide  a 
class,  and  that  one  must  restrict  the  coefficients  to  some 
class  Intermediate  between  this  and  the  tamer  class  of  periodic 
functions . 

4.  Nonlinear  Differential  Equations 

Whatever  the  difficulties  are  in  the  linear  case,  they 
fade  before  the  frightening  aspects  of  nonlinearity.  When 
treating  a  linear  equation,  we  enter  the  lists  armed  with  the 
fact  that  the  manifold  of  solutions  is  a  linear  space  of  finite 
dimension,  the  dimension  of  the  system.  When  dealing  with  non¬ 
linear  equations,  we  possess  no  such  advantage.  In  the  absence 
of  additivity,  the  superposition  principle,  the  standard 
methods  of  analysis  become  Inoperative  . 

New  techniques  must  be  employed.  One  is  the  idea  of  a 
regular  solution,  a  solution  finite  for  t  >  t^.  Tnis  very 
simple  and  fruitful  idea,  Introduced  by  Boral;  and  developed 
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by  Llndelof,  Hardy,  and  Fowler  (see  [2]  for  results  and 
references),  enables  us  to  study  In  great  detail  the  behavior 
of  Important  classes  of  solutions  of  polynomial  equations  of 
the  form 

( 1 )  g(u,u' ,t }  -  0, 
and 

(2)  u"*h(u,u',c). 

In  particular,  equations  of  the  Bnden— Powler  type, 

(3)  u"  +  uatb  *  0, 
can  be  intensively  analysed. 

/ 

Another  is  the  topological  approach  of  Poincare  which  we 
shall  discuss  oelow. 

5.  Nonlinear  Differential  Equations — Qeometrlc  Aspects 

To  Poincare  is  due  the  elegant  idea  of  using  fundamental 
topological  properties  of  curves  and  vector  fields  to  study  the 
properties  of  solutions  of  nornlnear  differential  equations. 

An  exce.ij.ent  exposition  of  these  ideas  is  given  in  [7]. 

Consider  tne  equation 

( 1 ,  u"  -  g(u,u'  )  , 

and  the  graph  of  a  solution  In  the  phase— plane,  the  iu,u' 
p 1 ane  . 
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As  pointed  out  Dy  Pulncare,  the  particular  solutions  which 
enable  us  to  sift  and  sort  the  diverse  kinds  of  solutions  of 
(1)  are  the  periodic  solutions,  which  In  the  phase  plane 
represent  closed  curves.  Assuming,  as  is  true  in  the  most 
Interesting  cases,  that  solutions  of  (1  are  uniquely  specified 
by  values  of  u  and  u'  at  some  time  t^,  we  see  that  the 
graphs  of  solutions  in  the  phase  plane  cannot  intersect  each 
other . 

Upon  this  simple  fact  hinge3  the  success  of  the  topological 
analysis  of  the  solutions  of  (!'.  As  sorn  as  we  turn  to  the 
study  of  fourth  c  rder  systems 

( 2  )  u'^y  «=  g(u,u',u'2;,u^'), 

or  to  the  study  of  twc  coupled  equations 

(!>)  uv  -g(u,u',v,v')» 

vl2  »  h  ; .  u , u '  ,  v  ,  v  '  '  , 
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a  more  natural  situation,  the  situation  changes  radically. 

In  four-Kiimenslonal  phase  space,  solutions  can  loop, 
intertwine  and  knot  in  the  most  intricate  and  abandoned 
fashion.  Such  is  the  degree  of  complexity  encountered  in  the 
study  of  general  curves  and  surfaces  in  four-dimensional  space 
that  far  from  expecting  topological  results  in  thlB  area  to 
contribute  to  the  study  of  the  particular  class  of  curves  and 
surfaces  generated  by  differential  equations,  one  should 
expect  the  contributions  to  flow  in  the  other  direction. 

6.  Discussion 

Little  is  known  *out  the  behavior  of  solutions  of  systems 
of  the  form  of  (6.5),  and  the  prospects  are  not  encouraging  for 
the  immediate  future.  Despite  intensive  research  over  the  last 
twenty  years,  no  breakthroughs  of  any  significance  have  been 
made . 

Sometime  in  the  future,  as  has  often  happened  in  the  past, 
a  theory  started  from  a  quite  different  origin  may  reach  a 
summit  from  which  we  shall  be  able  to  look  down  and  perceive  a 
number  of  paths  to  peaks  which  are  from  our  view  burled  in 
clouds.  Mere  likely,  we  may  never  reach  such  a  vantage  point. 
This  lack  of  mathematical  ability  may  affect  our  technological 
development  to  the  degree  that  other  techniques  based  upon 
different  mathematical  models  will  be  developed.  The  problems 
we  have  mentioned  concerning  linear  and  nonlinear  differentia] 
equations  may  ultimately  be  relegated  to  the  mathematical  limbo 
that  contains  so  many  problems  of  live  interest  in  the  eighteenth 
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and  nineteenth  centuries. 

If  the  present  deadlock  continues,  we  will  be  forced  to 
the  reluctant  conclusion  that  the  classical  vein  of 
differential  equations  is  played  out.  Fortunately,  the  modem 
theory  of  control  processes  provides  a  natural  continuation 
with  unlimited  regions  for  research  at  the  moment. 

This  is  another  example  of  the  by  now  fairly  widely 
recognized  fact  that  pure  mathematics  without  the  constant 
stimulation  and  enlargement  by  the  physical  world  faces  the 
dre?r  prospects  of  sterility  and  decadence. 

7.  Nonlinear  Differential  Equations — Computational  Aspects 

The  Inadequacy  of  our  analytic  techniques  In  the  face  of 
nonlinearity,  and  perhaps  even  the  non-existence  of  such 
techniques,  compels  us  to  focus  the  greater  part  of  our 
attention  upon  the  computational  solution  of  differential 
equations . 

Here,  we  face  the  problems  of  feasibility,  accuracy  and 
stability.  The  overall  question  is  that  of  adapting  an  analytic 
algorithm  such  as  successive  approximations  to  the  task  of 
supplying  numerical  answers  to  numerical  questions. 

An  enormous  amount  of  research,  and  mathematical  experi¬ 
mentation,  Is  required  in  this  area.  It  must  be  realized  that 
significant  computational  advances  can  only  be  consequences  of 
significant  analytic  advances.  The  two  go  hand-in-hand,  with 
one  acting  as  an  incentive  to  the  other. 

Let  us  discuss  briefly  two  of  the  new  techniques  developed 


P-1602 

9-23-59 

-12- 


over  the  last  few  years.  One  la  the  concept  of  quaalllnearlza 
tlon ,  [8]  ,  [9j  ,  ana  the  other  is  triat  of  sequential 
Computation  f  (1(3  . 

To  indicate  the  basic  Idea  of  quasilinearization,  let  us 
consider  the  problem  of  solving  the  two  point  boundary— value 
problem 

(1)  u"  +  g(u)  -  0. 

u(0)  «  u(a)  «  0. 


Let  us  suppose  that  a  is  small  enough  so  that  there  is  a 
unique  solution,  and  that  g(u)  possesses  a  second  derivative. 
Let  u0  be  an  initial  approximation,  and  let  the 

sequence  [u  !,  u  •  1,2,...,  be  generated  by  means  of  the 

l  n  j 

equation 


(2) 


un  +  8<un-l>  +  (un  "  un-l ^ ' lun-l >  ’  °> 
1  n  (0  )  "  Un(&)  "  0* 


It  can  be  shown  that  with  this  method  of  successive  approxima¬ 
tions  that  the  convergence  is  quadratic ,  l.e., 

2n 

|u  —  u  . |  -  0(6  ,  rather  than  geometric,  !u  -  u  -I  -  0(6  ), 

and,  in  some  cases,  that  it  Is  also  monotonic .  Por  details, 
see  [9]  • 

This  method  has  also  been  applied  successfully  to  different 
types  of  partial  differential  equation;: ;  see  [9],  [  1 1  ] . 

The  concept  of  sequential  computation  is  related  to  the 
existing  theory  of  computation  in  the  very  came  way  that 
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sequential  analysis  compares  to  statistical  theory  prior  to 
Wald.  At  the  present  time,  when  considering  an  equation  Buch 

d£> 

(3)  ^  -  g(u>,  u(0)  -  c, 


we  generally  use  a  dlsci’ete  version  such  as 

-Sa---  •  g(V»  uo  *  c> 


where  un  *  u(n&).  if,  over  certain  portions  of  the  t— Interval, 
the  function  u(t)  exhibits  too  wild  a  behavior,  e.g., 


u 


we  decrease  the  grid  size  over  this  interval.  This  is  usually 
done  in  an  a  posteriori  fashion.  By  this  we  mean  that  first 
we  run  through  the  solution  with  a  grid  size  A,  and  then 
begin  to  worry  if  we  observe  a  behavior  such  as  that  shown 
above . 

What  would  be  much  more  efficient  would  be  to  have  a 
computational  technique  which  automatically  adapts  Itself  to 
the  behavior  of  the  solution  that  is  being  computed.  One  way 
of  doing  this  would  be  to  make  L  dependent  upon  the  slope  and 


the  curvature  of  the  solution.  In  place  of  a  recurrence 
relation  such  as  (4),  we  would  have  two  relations  such  as 
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(5) 


u 


n+1 


—  u 


n— 1 


-  g(u 


n 


n 


-  c 


An  ‘  h(VVl'Un-2)- 

Por  some  further  details,  see  [1*0  . 

This  is  merely  one  aspect  of  the  general  concept  of 
adaptive  algorithms  rather  than  fixed  algorithms.  The  area  is 
almost  completely  unexplored. 


8.  Linear  Differentia^  Equations — Dimensionality  Difficulties 
Leaving  for  the  moment  the  rocky  region  of  nonlinearity, 
let  us  discuss  a  major  problem  associated  with  linear 
differential  equations. 

Suppose  that  we  have  reduced  the  problem  with  which  we 
are  concerned  to  that  of  determining  the  solution  of  the  linear 
differential  equation 

(1 )  3T  "  Ax»  *  c » 

where  A  is  a  constant  matrix.  If  the  dimension  of  x  is 
moderate,  say  ten  or  twenty,  we  c:.n  consider  the  solution  to 
be  routine.  If  the  dimension  is  large,  say  several  hundred  or 
a  thousand,  then  we  have  dimensionality  difficulties. 

There  are  now  several  different  techniques  that  we  can 
employ  to  replace  a  direct  method  of  solution  based  upon  the 
explicit  analytic  representation. 
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In  the  first  i- lace,  we  can  examine  the  original  problem 
and  see  whether  or  not  it  can  be  reformulated  in  such  a  way  aa 
to  lend  itself  to  a  computational  solution  in  terms  of  vectors 
of  considerably  lower  dimension.  In  a  number  of  cases,  this 
can  be  done,  see  [12]  ,  [13]  ,  .  Now  that  we  are  seriously 

beginning  to  consider  systems  of  quite  high  dimension,  it  Is 
essential  that  the  mathematical  models  employed  in  the  past 
for  systems  of  low  dimension  be  systematically  examined  from 
the  standpoint  of  computational  and  analytic  convenience.  We 
shall  return  to  this  matter  below. 

The  basic  idea  is  that  of  reducing  a  process  of  high 
dimension  by  a  sequence  of  processes  of  low  dimension.  This 
is  the  basic  idea  of  dynamic  programming,  [|2  ,  and  is  the 
essence  of  a  new  method  due  tc  Kron,  the  method  of  "tearing,’* 
whicn  seems  to  have  very  interesting  possibilities;  see  Roth,  [l 6^  . 

Along  similar  lines  are  the  relaxation  methods  of 
Southwell,  and  the  "flooding  method"  of  Poldyreff,  f.'Q  .  Much 
remains  to  be  done  to  make  these  techniques  generally  applicable 
and  efficient.  One  has  the  feeling  that  mathematicians  have 
really  understood  the  nature  of  the  processes  with  which  we  deal 
on  the  whole  so  clumsily,  the  analytic  and  computational  algor¬ 
ithms  will  consist  of  very  eimple  operations,  the  proof  of  whose 
efficacy  will  depend  ufon  quite  sophisticated  concepts.  At  the 
present  time,  we  largely  depend  upon  very  complex  operations 
resting  upon  very  rudimentary  concepts. 


jLl  The  Inverse  Problem 
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So  far  we  have  assumed  tnat  the  equation  is  given  and  that 
it  is  required  to  determine  the  behavior  of  the  solution  over 
time.  In  many  situations,  the  reverse  is  the  case.  Given  the 
behavior  over  time  of  a  vector  x(t;,  it  is  required  to  deter¬ 
mine  the  possible  equations  which  can  produce  this  behavior. 

This  problem  has  been  intensively  investigated  in 
connection  with  Sturm— Llouviile  equations,  (see  M.  M)  and 
is  closely  connected  with  the  "black  box"  problem  we  shall 
discuss  subsequently. 

_10  Time-lags  and  Hereditary  Effects 

We  meet  all  the  foregoing  problems  and  some  additional 
ones  if  we  take  into  account  delay,  time— lags,  retardation*, 
and  he  red  i  tar/  effects  In  genera. 

The  simplest  equations  arising  lr  this  way  are  the 
dlfferential-alfferenec-  ^juatlons,  which  have  trie  f^rm 

(1  ^-g(x(t/,x(t-riy,...,K(t-rm<  . 

The  appropriate  initial  ccnditlon  is  riuw  an  interval  condition 

(2  x ( t ;  -  h ( t ) ,  0  <  t  <  rm . 

In  many  ca303,  more  complicated  equation*  are  necessary 


to  describe  the  state  of  the  system,  equations  such  as 
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(3)  -  g(x(t , , c/)tkl ( t , a  x(s'ia,...,  /5tk  (t,sjx(s)ds). 

°--oo  uu 

The  mu:t  Interesting  «.  f  these  ar»*  trie  ”erv»wul  equations 

v4  '  x(t)  =  g(t)  4-  Z*1  k  (t  -  3  x(s  da , 

°0 

arising  In  the  theory'  cf  queues,  counters,  lr.  the  theory  of 
branching  processes,  and  in  prediction  theory. 

Pox'  a  Jlscusalon  of  equations  vf  tills  nature,  tee  [2^,  [23]. 

11.  ftochastlc  Processes 

It  should  be  recognized  initially  that  the  formulation  of 
a  physical  process  in  deterministic  or  stochastic  terms  is  a 
matter  of  mathematical  convenience  rather  than  a  consequence 
of  the  Intrinsic  structure  of  the  process.  Let  us,  however, 
bypass  the  really  deep  problems  connected  with  the  advantages 
or  disadvantages  of  one  formulation  or  another,  and  instead 
Indicate  ..jme  o:  the  many  new  and  challeng4ng  problems  that 
arise  once  we  have  agreed  to  describe  the  behavior  or  a  circuit 
In  terms  of  differential  equations  with  stochastic  elements. 

We  shall  consider  only  ordinary  differential  equations, 
although  the  corresponding  problems  fcr  partial  differential 
equat’ona  are  of  great  importance  in  other  parts  of  mathe¬ 
matical  physics,  e.g.,  turbuience  theory. 

Let  us  consider'  four  particular  problems.  It  *ould  be  as 
easy  to  present  forty  problems,  since  the  field  is  practically 
unexplored.  A  certain  amount  of  effort  has  been  devoted  to 
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the  study  of  the  linear  equation 
( 1 )  L(u)  -  r(t ), 

where  L  Is  a  deterministic  operator  and  r(t)  Is  a  random 
function.  The  results  that  are  obtained  and  techniques  that 
are  employed  are  rather  straightforward,  and  thus  not  parti¬ 
cularly  challenging.  Far  more  Interesting  Is  the  situation 
In  which  the  operator  Itself  has  a  stochastic  structure. 
Questions  of  this  type  arise  In  the  consideration  of  circuits 
with  unreliable  elements,  or  In  the  study  of  circuits  subject 
to  external  Influences  of  Incompletely  known  type  which  affect 
the  structure  of  the  circuit.  For  an  Interesting  discussion 
of  results  obtained  to  date  for  the  case  of  linear  differential 
operators,  see  [?2j,  where  many  further  references  will  be 
found . 

In  order  to  illustrate  some  of  the  dl  fficultl  es  in  the 
study  of  this  problem,  let  us  consider  perhaps  the  simplest 
version,  that  of  a  linear  difference  operator. 

The  equation  we  wish  to  study  Is 

(  2 )  x  ( n  t  1  ;  f  ( n  ,  x  ( n  ) , 

where  x(n)  is  an  N- d 1  mens  1 ona 1  vector  and  Z(n)  Is  an  N  *  N 
random  matrix.  Lt  is  sufficient  to  consider  the  simplest  case 
where  the  '(n)  are  lnuependenl  random  matrices. 

It  is  not  dl  ‘  '.cult  to  obtain  analytic  expressions  for  the 
moments  or  the  components  of  x(n),  using  the  technique  of 


rlronecker  products  of  matrices,  see  [jj] ,  [34],  [25].  The  {'act 
that  Kronecker  products  of  matrices  bypass  the  difficulties 
Vndlced  Dy  none ommu tat  1  vl ty  of  matrices  has  not  been  suffi¬ 
ciently  exploited  in  the  study  of  linear  stochastic  systems 
cf  this  nature. 

Furthermore,  th*»re  is  little  difficulty  In  extending  these 
results  to  the  case  wher*»  the  Z(n)  are  correlated  in  some 
Markovian  fashion,  and  In  passing  to  the  limit  and  obtaining 
corresponding  resuits  for  linear  il fferentlal  equations, 
although  no  results  of  this  type  have  been  published. 

what  Is  reai Ly  difficult  Is  a  more  precise  determination 
of  the  asymptotic  behavior.  We  expect  the  solutions  to 
exhibit  exponential  Dehavlor,  and  the  moments  that  are  obtained 
Illustrate  this.  It  follows  that  it  is  the  moments  of  the 
logarithm  of  the  components  that  yield  more  detailed  informa¬ 
tion.  For  example,  we  would  Ilk**  to  be  able  to  find  the 
asymptotic  behavior  of 


(3)  Rxp(log  Xj (n ) )* 

as  n  — »  co  .  Here  x^(n)  Is  the  f 1 rst  component  of  x(n). 
Although  some  small  steps  In  this  direction  have  been  taker, 
see  [23],  no  satisfactory  methodB  or  results  exist  at  the 
moment . 

hor  our  second  problem,  let  us  turn  to  the  comparl  son  cf 


discrete  and  continuous  versions  of  stochastic  processes. 
Parenthetically,  let  us  observe  that  It  is  often  felt  that  the 
discrete  version  of  a  process  Is  an  approximation  Introduced 
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el  ther  for  computational  purposes,  as  when  we  replace  differ¬ 
ential  equations  by  difference  eiuatlons,  or  for  conceptual 
simplicity.  Tt  Is  worth  emphasizing  the  fact  that  the  reverse 
may  well  be  true,  and  certainly  Is  true  in  a  number  of  control 
processes.  The  actual  process  will  often  be  discrete,  but  a 
continuous  process  will  be  used  sometimes  for  analytic  conven¬ 
ience,  but  more  often  by  very  force  of  tradition.  If  one  takes 
Into  account  the  fact  that  the  solution  will  ultimately  be  ob¬ 
tained  by  means  of  a  digital  computer,  then  there  may  be  a  loss 
of  efficiency,  and  even  error.  Introduced  by  conversion  of  a 
discrete  process  Into  a  continuous  process  for  the  sake  of 
analysis,  and  then  by  reconversion  to  a  discrete  process  for 
the  sake  of  computation.  The  fact  that  the  original  discrete 
process  and  the  final  discrete  process  may  well  be  different 
processes  can  often  cause  grief.  On  the  other  hand,  this  may 
be  an  advantage.  Vie  do  not  wish  to  make  any  rigid  statement 
concerning  the  advisability  or  Inadvisability  of  one  procedure 
or  another.  *'hat  we  do  wish  to  emphasize  Is  that  It  Is 
essential  to  be  aware  of  these  matters,  so  that  one  can  follow 
the  procedure  which  Is  most  advantageous  In  any  particular 
process . 

It  Is  particularly  important  to  be  cognizant  of  these 
facts  at  the  present  time  since  the  digital  computer  has  Intro¬ 
duced  a  flexibility  Into  formulation  and  solution  of  mathemati¬ 
cal  problems  which  was  certainly  not  present  even  as  recently 
as  ten  years  ago. 
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flnce  we  Know  that  the  same  physica*  process  can  oe  formu¬ 
lated  In  many  dl  fferent  fashions,  discrete  or  continuous, 
deterministic  or  stochastic,  and  30  on,  It  Is  essential  that  we 
/now  the  relations  between  these  dll  rerent  formulations.  In 
particular,  It  Is  essential  that  different  formulations  yield 
the  same  numbers.  Perhaps  even  more  Important  would  be  a 
knowledge  of  when  tney  do  not,  since  this  would  be  a  most 
valuable  clue  concerning  the  true  nature  of  the  process. 

ror  the  case  of  deterministic  processes,  a  great  deal  Is 
Known  concerning  the  relation  between  discrete  and  continuous 
versions.  ror  example,  given  the  differential  equation 

(u)  37  g(x),  x(0)  -  c, 

and  the  discrete  approximation 

Vi  -  xn  =  x0  ’  c' 

wne  ’e  =  x(ni),  quite  simple  conditions  Imposed  upon  g(x) 
guarantee  that  x(n^)  — ►  x(t)  as  a  — »  ,  nA  —*■  t. 

Essentially,  these  are  the  same  conditions  that  ensure  exist¬ 
ence  and  uniqueness  of  solution  of  ( ^) ,  although  many 
re fl  nements  exist.  For  specific  resuits  and  further  reierences, 


Eo i  the  case  of  stochastic  processes,  this  problem  seems 
not  to  have  been  studied  at  all.  A  question  of  some  Interest 
Is  the  following:  -,'hat  *s  the  relation  between  the  solutions 


of  the  two  equations 
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('  '  X'  (l  )  =  g(x(t.  )  ,  r(l  ;  '  ,  x(f  )  e. 


and 


n-  1 


x 

n 


r 

n 


“  l 


X 


< 


"  9 


(x  =  x(nA),  r  -  r(riA^),  where  r(  t )  Is  a  random  function 
with  a  given  distribution?  If  r(t)  Is  uniformly  bounded, 
there  Is  no  difficulty  In  applying  the  usual  methods.  The 
Interesting  case  Is  that  where  r( t )  Is  unbounded. 

Let  us  turn  bacK  to  the  "black  box"  problem.  Given  a 
linear  operator,  L,  such  as  a  linear  differential  operator, 
we  can  characterize  It  In  simple  fashion  by  examining  the 
solutions  o  *'  the  Inhomogeneous  equation 


iu>t 


as  r. »  ranges  over  a  se'  of  values.  .Vhat  does  one  do  for  a 
nonlinear  d!  f ferenti  a  operator"  This  Is  a  problem  which  has 
been  studied  to  some  extent,  see  w.  ;h. 

's  It  true  that  nonlinear  operators  are  more  appropriately 
studied  cy  means  of  stochastic  forcing  terms  than  by  means  of 
deterministic  ones.  For  example,  should  one  use  the  equation 

(9)  N(u)  -  r(t  ) 

where  r(t  Is  a  random  function  with  an  appropriate  distri¬ 
bution,  rather  than  an  equation  o'-  the  form 

N  ( u 


(-  ) 


9 
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or  an  equation  with  another  type  of  deterministic  forcing  term? 

Closely  reiated  to  this  problem  Is  the  f'lnai  proolem  we 
wl3h  to  pose.  The  Importance  or  the  Gaussian  distribution  lies 
In  Its  relative  Invariance  under  a  linear  transformation.  By 
this  we  mean  that  If'  x  and  y  are  both  Gaussian,  then 
x  ♦  y  Is  Gaussian.  Another  way  01  looking  at  this  Is  that  the 
linear  transformation 

(  i  1  )  T  (  x  )  =  x  ♦  r 

Is  a  particularly  simple  one  to  study  when  r  is  Gaussian. 

Viewed  in  this  way,  the  natural  question  to  ask  13:  -Vhat 
class  of  distributions  should  be  used  when  we  study  more 
general  nonlinear  transformations  of  the  form 

(12)  T(x)  -  g(x,r)" 

Furthermore,  what  type  of  limiting  distributions  should  oe 
expected  when  the  transformation  Is  Iterated  repeatedly" 

these  questions  seem  of  great  difficulty.  It  may  well  be 
tnat  the  way  to  approach  them  Is  to  do  some  mathemat’cal  experi¬ 
mentation  with  digital  computers,  us’ ng  particular  types  of 
slmp.e  i  rans format  1 ons  and  random  variables.  <vork  ol  this  type 
has  been  carried  ^ut.  by  "'teln  and  U1  in  at  jo s  Alamos,  but  not 
put 1 1  shed  as  yet. 


CONTROL  PROCESSES 
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1.  Control  Processes 

Let  us  now  leave  the  realm  of  descriptive  processes  and 
turn  to  the  study  of  control  processes.  There  Is  little  point 
at  the  present  time.  In  view  of  the  many  different  uses  of 
these  words  In  many  diverse  fields.  In  attempting  to  make 
precise  what  Is  meant  by  the  term  "control  process."  Let  ua 
simply  and  Intuitively  say  that  we  wish  to  use  It  to  describe 
any  physical  process  In  which  the  original  behavior  of  the 
system  la  influenced  In  some  way  in  order  to  force  It  to 
operate  In  a  more  desirable  fashion. 

A  priori.  It  might  be  Imagined  that  the  study  of  control 
processes  Is  more  complex  than  that  ol  descriptive  processes. 

To  some  extent,  this  Is  true.  On  the  other  hand,  the  addi¬ 
tional  information  fum’ shed  by  the  requirement  that  we  exert 
control  In  a  mo3t  efficient  fashion  orten  yields  a  considerable 
simplification.  It  Is  generally  the  case  that  optimal  policies 
are  simpler  than  obvious  feasible  policies,  once  their  true 
nature  Is  understood. 

It  may  very  we.,  te  the  case  that  our  excursion  Into  this 
more  difficult  *'leld  wLi  ultimately  yield  the  vantage  points 
required  to  re30ive  some  o'-  the  truj.y  obdurate  problems  In  the 
theory  of  descriptive  processes. 

In  any  case,  the  theory  of  control  processes  Is  replete 
w*  tl  fascinating,  challenging  mathematical  problems  of  the 
utmost  significance  to  contemporary  society.  ei/hat  more  can  a 


mathematician  asK" 


2.  Variational  Analysis 
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Cnce  again  beginning  In  a  traditional  way,  let  us  consider 
the  following  analytic  version  of  a  feedback  control  process. 
Consider  a  physical  sysler,  3  whose  state  at  time  t  Is  des¬ 
cribed  by  the  vector  x(t)  which  satisfies  a  differential 
equation 

(1)  ^  *  g(x,y),  x(0)  -  c. 

Here  y  =  y(t)  Is  the  control  vector  which  Is  to  be  chosen  so 
as  to  minimize  the  functional 

(?)  J(x,y)  =(/"T  h(x,y)dC(t). 

0 

I'  dG(t)  ---  g(t)dt,  we  have  a  variational  problem  of  the  usual 
type.  If  0(t)  !o  a  s tep- funct 1  on  with  a  single  Jump  at  T, 
we  have  the  problem  o°  mini'll  zing 

(j)  J(x.y)  =  h(x(T),y(T)), 

a  te  ml  na  i  cont  ro  1  problem.  In  many  cases,  we  have  a  combina¬ 
tion  of  both  types. 

when  these  prooierr.s  are  an  reached  by  the  classical  methods 
of  the  calcuius  of  variations,  they  lead  to  nonlinear 
d!  I' n*-  rent  1  a  1  equal ‘oris  of  dimension  double  the  dimension  of  x, 
w  th  two- point  boundary  conditions.  These  problems  are  analy¬ 
tically  Intractable  and  require  a  great  deal  of  effort  and  good 
fortune  If  a  reliable  numerical  nsult  Is  desired. 

A  new  approach  to  questions  of  this  nature  along  the  *ines 
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of  dynamic  programming,  10  ,  13  ,  permits  some  advances  to  be 

made.  However,  for  problems  of  even  moderate  dimension  any 
routine  application  of  these  techniques  also  runs  Into  acute 
computational  dl'flcultles. 

We  can  consider  the  problem  of  obtaining  feasible  algor¬ 
ithms  for  the  computational  solution  of  realistic  feedback 
control  processes  to  be  one  of  the  challenging  and  Important 
fields  of  research  In  the  Immediate  future.  Whatever  the 
methods  In  use  now,  better  ones  are  needed. 

We  shall  have  more  to  say  on  the  subject  below. 

3.  Constraints 

To  turn  the  screw  a  11  tt.  '•  p  tighter,  let  us  point  out  that 
in  many  significant  situations,  /-*»  do  not  have  complete  free¬ 
dom  In  a  choice  of  y.  The  simplest  and  most  natural  type  of 
constraint  is  one  of  the  fom- 

(1)  |y,  <;  m,,  1  -  1,2,..., N,  £  t  £  T. 

There  Is  now  considerable  difficulty  In  applying  variational 
analysis,  and  even  the  most,  simply  posed  analytic  problems 

f  •  r  - 

possess  obstacles  In  the  path  of  their  solution;  see  [1C  , 

It  would  r>e  extreme!,  y  valuable  to  have  available  the 
complete  solution  to  certain  prototype  problems.  For  example, 
consider  the  problem  of  min' ml  zing  the  functional 

(2)  J(x,y)  =/  T  f ( x, bx )  -  (y,Cy)l d0(t ) 

c0  1  J 


over  al  .  y  where 


(3) 


/.  4 

—  -  -x  ,  y. 


x(0)  -  c, 


f-  : 


9  2; -5. 


-27- 


and  y  Is  subject,  to  (i  .  J.ui  this  -  .  i*u.  b  be  completely 
re so i ved' 


4.  Bang- uang  Control 

A  very  important  type  o:'  problem  that  arises  Is  many 
areas  is  th3t  of  forcing  a  system  back  to  an  equilibrium 
position  in  a  minimum  time.  A  great  deal  of  effort  has  been 
devoted  to  the  study  of  linear  systerrs,  such  as  that  of  (3-i), 

with  constraints  of  the  form  of  (3-i),  and  the  problem  has 

r  ’  '  ’ 

been  fairly  well  tamed;  see  [3Q,  31  ,  where  further  references 

may  be  found. 


For  nonlinear  syBtems  of  small  dimension,  the  functional 


equation  technique  of  dynamic  programming  may  be  employed,  see 
r 

[31J .  For  nonlinear  systems  of  moderate  or  high  dimension,  no 
technlqt.es  exist  for  either  analytic  or  computational  solution 
at  present. 

Problems  of  this  type  are  actually  only  particular  cases 
of  variational  problems  f  1  rrq  .  1  c  1 1  nature.  ?hf3e  are 
characterized  by  the  property  that  the  form  of  the  criterion 
function  Itself  depends  upon  the  policy  that  Is  pursued. 

For  example,  'n  place  of  a  fixed  functional  of  the  form 


0) 


J(y ) 


fb- 


h(x,y  jat, 


we  may  envisage  one  of  the  ’orm 


P-  u 

4-0  K 

-  28- 


(2)  J(y  )  >  /;T^X#>  h(x,y  )dt . 

1  o 

Bang-tang'1  control  corresponds  to  h(x,y)  5  ]t  which  means 
that  J(y)  measures  the  duration  of  the  process. 

p  .  Successive  Approximation 

The  standard  analytic  technique  for  circumventing  and 
overcoming  the  usua '  intractable  aspects  of  the  equations  of 
analysis  Is  the  method  ol  successive  approximations.  Although 
this  method  has  been  applied  with  perseverance  and  success  to 
the  study  of  descriptive  processes,  for  some  reason  or  another 
little  has  been  done  with  it  In  the  study  o f  control  processes. 

From  the  very  nature  of  the  feedback  control  process,  one 
would  suppose  that  successive  approximations  would  be  Ideally 
suited  to  the  study  of  these  processes. 

Fet  us  then  base  our  method  of  successive  approximations 

around  this  fact.  tartlng  with  some  Initial  approximation  to 

0  0 

a  policy,  y',  and  an  approximation  to  the  state  vector,  x  , 

let  us  replace  J(y)  by  Its  expansion  around  these  functions 
x  and  yJ.  In  this  way,  reia'nlng  only  terms  up  to  and 
Including  the  second  decree,  we  obtain  a  new  functional  which 

Is  quadratic  in  x  and  y.  similarly,  let  us  expand  the 

0  C 

(unct’on  g(x,y)  around  the  point  x  ,y  ,  retaining  only 
terms  up  to  the  first  degree.  in  this  way  the  describing 
^luations  are  linearized. 

.'oiving  this  approximate  variational  problem  In  analytic 
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terms,  we  obtain  a  new  policy  vector,  y1,  and  a  new  state 
vector  x1.  This  process  cam  now  be  continued.  We  now  face 
the  usual  problems  concerning  convergence,  rate  of  convergence, 
convergence  to  a  relative  Instead  of  absolute  extremum,  and  so 
on. 

There  are  a  number  of  other  ways  in  which  the  concept  of 
successive  approximations  can  be  applied;  see  [ 9  J ,  10 J ,  !.32j  , 
[l5J.  A  quite  different  approach  will  be  described  below. 


6.  Approximation  to  Functions 

Let  us  continue  on  this  theme  of  dimensionality.  We  have 
been  forced  to  think  of  various  type3  of  approximations  because 
of  the  Impossibility  of  dealing  with  functions  of  a  large 
number  of  dimensions  In  any  direct  fashion.  Because  of  the 
limited  fast  memory  of  present-day  digital  computers,  and  of 
any  that  we  can  contemplate  in  the  Immediate  future,  we  must 
renounce  any  routine  approach  based  upon  the  tabulation  of 
functions  of  say  ten  variables  over  the  grldpoints  of  a  ten- 
dlmenslonal  region. 

Instead,  If  we  are  dealing  with  smooth  functions,  we  can 
contemplate  a  more  analytic  description  in  terms  of  polynomials 
or  in  terms  of  orthonormal  expansions.  These  latter  are 
preferable  since  they  usually  possess  super’ or  cc  ivergence 
prop  erties . 

If  we  use  an  approximation  of  the  form 


N 


£  a  v  ( s )v  (t ) 


m,n-0 


m,n  m'  '  n' 


(1) 


u(s,t) 
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over  the  unit  square,  we  require  N(N  +  l)/2  coefficients  to 
determine  the  function  u(s,t).  If  we  tabulate  the  values  of 
the  function  at  the  gridpoints  (kA,i/A),  we  require  l/A2 
values.  As  we  Increase  the  dimension  of  the  function  u,  the 
number  of  grldpolnts  goes  up  exponentially,  while  the  number 
of  coefficients  required  to  determine  the  function  to  the  same 
degree  of  accuracy  by  means  of  an  approximation  of  the  form  In 
(l)  goes  up  at  a  much  siower  rate--provided ,  of  course,  that 
the  function  Is  smooth.  In  many  important  applications,  this 
will  be  the  case. 

It  Is  Important  to  note  that  this  type  of  approximation 
can  be  used  not  only  in  the  study  of  control  processes,  but 
In  the  study  of  descriptive  processes  as  well.  A  brief 
discussion  of  a  particular  control  process  treated  in  this 
fashion  will  be  found  In  [  . 


’.  Stochastic  Control 


As  soon  as  we  turn  to  the  subject  of  stochastic  control 


processes,  we  raee  the  problem  of  treating  variational 


questions  Involving  stochastic  functionals  ar.d  stochastic 
differential  equations. 

The  functional  equation  technique  of  dynamic  programming, 
[•to] ,  can  be  readily  applied  to  treat  these  new  types  of 

analytic  problems.  As  a  matter  of  fact,  the  treatment  is 
abstractly  similar  to  that  of  deterministic  control  processes. 

There  are,  however,  a  large  number  of  mathematical 
questions  In  this  field  which  have  neither  been  precisely 


P-lo  _ 
q  .  o  -  -  ",  -> 

isked  nor  precisely  answered.  (Generally,  we  want  a  rigorous 
I  jmiulatlon  of'  th-_-  j '  o  I  ■=*  :n  of  ml  r:l  rd  z  l  ng  the  functional 

'  1 

(  i  )  J(y  )  -  -xt  ^  '  1  g(x,y ,r)dt 

i‘  L  '  J 

where  x  arid  y  are  connected  by  means  of  a  stochastic 
di  fferent.  iai  equation 

(  d,  -*’i(  *,.r  ,  r  '  ,  x{C  \  =  c  . 

Here  Hxp  represents  the  expected  value  taken  with  regard  to 
r 

the  random  elements. 

As  usua i ,  It  would  he  worthwhile  to  begin  with  to  consider 
the  case  o;  quadratic  functionals  subject  to  linear  describing 
equations,  since,  as  mentioned  above,  these  ea..  be  used  as  the 
point  ol'  departure  for  successive  approximations. 

If  would  aiso  t'e  useful  to  treat  the  "bang- bang'  control 
probj-ru  unuer  the  assumption  that  the  describing  equation  <s 
i 1  near , 

(  j  ’  ^  =  Ax  t  y  ^  r,  x  ( 0  )  -  c , 

ano  *v  w 1  sh  U  m  n!  n!  e  the  expected  t '  me  to  get  to  the  origin. 
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ADAPTIVE  PROCESSES 


1.  Preliminaries 

We  have  briefly  discussed  descriptive  and  control  aspects 
of  deterministic  and  stochastic  processes.  In  the  main,  the 
mathematical  questions  that  we  have  asked  have  been  easy  to 
pose,  but  difficult  to  answer.  This  Is  rather  typical  of 
classical  analysis.  In  the  next  two  parts  of  the  paper,  we 
wish  to  depart  from  thfe  well-charted  roads  and  follow  some 
brambly  trails.  Our  aim  is  to  discuss  some  parts  of  nonlinear 
circuit  analysis  which  give  rise  not  only  to  questions  which 
are  difficult  to  answer,  but  difficult  even  to  ask.  As  a 
matter  of  fact,  to  know  what  to  ask  and  how  to  ask  it  is  to  be 
far  along  the  path  to  a  solution. 

Basically,  we  wish  to  consider  classes  of  problems  which 
are  not  well-defined  according  to  conventional  tenets.  One 
way  of  formulating  problems  of  this  nature  precisely  is 
furnished  by  the  theory  of  games,  [35],  [36j ,  either  single-stage 

r  i 

or  multi-stage,  [15J .  We  shall  utilize  the  multi-stage  aspect 
to  provide  another  formulation.  It  must  be  emphasized  that 
there  1  s  no  definitive  way  of  handling  these  new  types  of 
problems  at  the  present  time,  and  it  is  extremely  unlikely 
that  there  ever  will  be. 

The  processes  we  wish  to  discuss  are  those  in  which  we 
lack  certain  essential  Information  which  was  given  to  us  in 
our  study  of  deterministic  and  stochastic  processes.  This 
information  can  be  obtained,  bit  by  bit,  as  the  process  un- 
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folds.  Not  only  must  wo  make  decisions  on  the  basis  of  the 
Information  we  possess,  but  we  mu3t  make  decisions  as  to  what 
Information  we  wish  to  possess.  Processes  of  this  nature,  we 
call  "adaptive  processes."  Occasionally,  the  term  "learning 
process"  Is  employed. 

In  3ome  of  these  processes.  It  may  be  senseless  to  speak 
about  "optimal  policies,"  but  quite  sensible  to  talk  about 
feasible  or  efficient  policies;  cf.,  for  example,  [37]. 

2.  Unknown  Distributions 

Rather  than  continue  In  these  vague,  foreboding  terms, 
let  us  consider  a  stochastic  control  process  governed  by  the 
equation 

U)  xn+1  -  g(Vyn,rn),  X0  =  c, 

where  ir  l  Is  a  sequence  of  random  variables  with  an  unknown 
'  nj 

distribution. 

What  do  we  mean  by  an  optimal  control  policy  In  a  situa¬ 
tion  like  this?  One  way  out  Is  to  employ  the  theory  of  games, 
under  the  assumption  that  some  malevalent  Influence  Is 
choosing  the  unknown  distribution.  This  Is  unduly  pessimistic, 
and.  In  any  case,  eliminates  the  possibility  of  our  learning 

about  the  distribution  of  r  as  the  process  continues. 

n 

The  question  that  faces  us  now  Is  that  of  formulating  this 
learning  process  In  mathematical  terms.  We  can  proceed  In 
various  ways: 
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(2)  (a)  We  can  record  the  past  history  of  the  process, 

^xk'rk|'  k  »  1,...,—  M,  and  use  this  data  to 
predict  the  distribution  of  rQ,  and  so  on. 

(b)  We  can  use  an  a  priori  distribution  for  r^, 
dG(r),  and  use  a  fixed  rule  to  modify  dO(r) 
on  the  basis  of  observation  of  rQ  or  x^. 

(c)  We  can  use  an  a  priori  distribution  dO(r,a), 
dependent  on  a  finite  set  of  parameters,  ^a  ,, 
and  use  a  fixed  rule  to  modify  a  on  the 
basis  of  observation  of  r^  or  x^, 

and  so  on,  and  so  on. 

Not  only  do  we  face  the  task  of  Justifying  a  choice  of 
one  model  or  another,  but  granted  any  particular  formulation, 
we  must  endure  formidable  mathematical  difficulties  in  our 
attempts  to  use  these  mathematical  models  to  obtain  numerical 
answers  to  numerical  problems.  In  some  cases,  the  functional 
equation  technique  of  dynamic  programming  can  be  utilized;  see 

iioi.  N.  >sj,  M-  For  an  interesting  application  to  pre- 

r  i 

diction  theory,  see  4lJ .  . 


slon  of 


is  known. 
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(c)  that  the  criterion  for  determining  an  optimal 
policy  Is  not  precisely  known. 

(d)  that  the  duration  of  the  process  Is  not 
known. 

The  domain  or  the  new  theory  of  adaptive  processes  con¬ 
sists  of  problems  of  this  nature,  and  contains  some  other 
troglodytes  which  we  shall  dig  out  below.  Some  of  these 
questions  are  discussed  In  [l<^ . 

It  seems  reasonable  to  expect  that  these  totally  new 
types  of  problems  will  require  new  methods  and  new  concepts. 
One  might  surmise  that  the  emphasis  will  be  more  upon  policies 
and  less  upon  functions,  and  more  upon  approximation  In  policy 
space,  than  In  manipulation  with  functions  and  functionals. 


THE  CURSE  OF  DIMENSIONALITY 
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1.  Introduction 

Examining  the  various  problems  that  we  have  skimmed  over, 
lamenting  all  the  while.  It  Is  clear  that  the  conceptual, 
analytic  and  computational  features  become  enormously  more 
complex  whenever  the  dimension  of  the  system  S  becomes  large. 
This  is  an  example  of  the  metaphysical  axiom  that  a  signifi¬ 
cant  difference  In  quantity  constitutes  a  difference  In 
quality . 

In  this  concluding  section,  we  wish  to  touch  lightly  upon 
some  new  techniques  that  have  been  proposed  In  recent  years  to 
treat  various  problems  Involving  large  numbers  of  state  vari¬ 
ables,  vast  accumulations  of  data,  and  lack  of  basic 
Information. 

To  our  mind,  this  field  is  the  most  challenging  of  all 
the  new  areas  that  have  opened  up  over  the  last  cen  years. 

It  is  well  suited  for  the  young  research  scientist  since  it 
requires  new,  original  and  daring  ideas  with  no  apparent  pay¬ 
off  for  erudition  or  experience.  If  anything,  saturation  with 
old  ideas  Is  a  handicap. 

Finally,  the  fundamental  nature  of  these  problems  makes 
every  contribution  a  contribution  to  a  half  a  dozen  fields  at 
the  same  time. 

2.  Too  Much  Data 

Much  of  our  previous  discussion  has  centered  upon  the 


difficulties  encountered  in  the  study  of  processes  which  are 
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incompletely  described  from  the  classical  point  of  view.  It 
is  important  then  to  note  that  in  treating  processes  of  high 
dimension,  involving  large  quantities  of  data,  complete  infor¬ 
mation  is  as  much  of  a  handicap  as  a  scarcity  of  information. 

As  a  matter  of  fact,  the  two  problems  are  intimately  related, 
as  we  shall  now  explain. 

In  using  information,  we  must  take  into  account  the 
problems  Involved  in  storing  It,  In  retrieving  it  when  desired, 
in  using  it  in  an  efficient  fashion,  and,  finally,  in  reckon¬ 
ing  with  the  cost  of  using  the  information,  and  the  time 
involved.  Ideally,  the  last  two  should  be  part  of  the  question 
of  efficient  use. 

It  is  clear  that  in  many  cases  this  means  that  we  will 
not  have  the  time  to  examine  all  the  data.  Consequently,  we 
will  be  in  the  position  of  rejecting  information,  which  means 
that  we  are  back  in  the  situation  of  Incomplete  information. 

The  general  problem  of  recognizing,  evaluating  and  stor¬ 
ing  Information  is  one  of  the  major  problems  in  current 
science,  and  one  of  the  major  problems  of  our  civilization. 

It  would  be  Important  to  construct  various  mathematical  models 
which  study  the  rejection  of  large  quantities  of  information. 

A  combination  of  the  theories  of  mathematical  statistics  and 
differential  equations  may  yield  some  useful  results.  We 
shall  discuss  a  particular  model  of  this  type  below. 

3.  Analogy 

What  we  have  mentioned  above  opens  up  an  interesting 
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channel  of  Investigation.  This  Is  the  Idea  of  proceeding  by 
analogy .  To  some,  use  of  this  Idea  is  implicit  In  the  studies 
of  stochastic  and  adaptive  systems  which  we  have  mentioned 
above.  Let  us  now  attempt  to  examine  the  idea  in  more  detail. 

Faced  with  the  prospect  of  utilizing  a  physical  system  S 
of  such  complexity  that  we  cannot  employ  the  actual  equations 
describing  the  state  of  the  system,  or  in  ignorance  of  these 
equations,  we  construct  another  system  S',  described  by 
fewer  state  variables  and  simpler  equations,  and  use  this 
system  as  the  basis  of  our  decisions. 

It  Is,  of  course,  necessary  from  time  to  time  to  check 
the  conclusions  based  upon  S'  with  results  obtained  from 
observation  of  S,  and  to  make  changes  either  in  the  conclu¬ 
sions  or  In  the  structure  of  S'  on  the  basis  of  this 
comparison.  It  is  well  to  point  out  that  this  comparison  may 
not  be  as  easy  to  effect  as  might  be  expected  a  priori,  since 
the  Interpretation  of  the  observation  of  S  may  Itself  depend 
upon  the  artificial  system  S'  which  we  tve  constructed. 

It  is  very  possible  that  the  human  brain  operates  in  some 
fashion  as  that  described  above.  The  problem  that  we  wish  to 
pose  is:  "How  do  we  construct  mathematical  models  of  processes 
carried  out  In  this  fashion?" 

4.  The  Techniques  of  Box 

It  Is  appropriate  in  this  context  to  mention  the  Intere¬ 
sting  and  effective  methods  proposed  by  Box  and  his  colleagues, 
[•+«],  [*3],  [44].  one  of  the  problems  that  he  tackles  Is  that  of 
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controlllng  a  chemical  engineering  process  where  the  precise 
chemical  and  physical  Interactions  are  not  known. 

5.  The  Tearing  Technique  of  Kron 

Another  new  and  promising  technique  Is  that  put  forth  In 
recent  years  by  Kron,  M. M.  He  provides  systematic 
techniques  for  decomposing  a  complex  system  Into  a  set  of 
systems  of  simpler  type. 

6.  Sequential  Search 

Finally,  let  us  mention  the  attempts  lr.  recent  years  to 
tackle  In  some  direct  fashion  the  problem  of  determining  the 
maximum  or  minimum  of  a  function  of  a  large  number  of  variables. 
For  the  reasons  which  we  have  previously  discussed  It  is  clear¬ 
ly  Impossible  to  accomplish  this  In  any  simple  enumeratlve  way. 

For  the  case  of  functions  of  one  variable,  some  very  ele¬ 
gant  results  exist,  see  [j»8]#  49],  ;5oj.  For  functions  of  two  or 
more  variables,  optimal  policies  seem  quite  difficult  to  obtain, 
and  It  Is  reasonable  to  suppose  that  simple  sub-optimal  policies 
would  be  as  efficient  in  practice.  However,  even  these  seem 
hard  to  find. 

7.  Routing  and  Switching 

We  have  avoided  all  discussion  of  the  very  Interesting 
combinatorial  questions  that  arise  in  the  study  of  optimal 
routing  in  communication  and  travel  networks.  The  reader  con¬ 
cerned  with  these  fields  may  wish  to  refer  to  the  expository 
papers  [51],  [52],  where  many  additional  references  can  be  found. 
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